Supporting automation of customer service

ABSTRACT

The present disclosure describes various methods, computer-readable media, and apparatuses for supporting customer service automation. The support for automation of customer service may be based on analysis of conversations between customers and customer service agents of a customer service center. The support for automation of customer service may be based on analysis of conversations between customers and customer service agents to identify conversation-level topics and utterance-level topics from the customer service conversations. The support for automation of customer service may be based on use of conversation-level topics and utterance-level topics identified from the customer service conversations to control initiation of customer service automation actions for supporting automation of customer service of the customer service center. The support for automation of customer service may be based on use of conversation-level and utterance-level topics identified from the customer service conversations to control selection, and design, of chatbots for the customer service center.

TECHNICAL FIELD

This application is a continuation of U.S. Pat. Application Serial No.16/792,987, filed on Feb. 18, 2020, now U.S. Pat. No. 11,544,721, whichis herein incorporated by reference in its entirety.

The present disclosure relates generally to customer service, and moreparticularly to methods, computer-readable media, and apparatuses forsupporting automation of customer service.

BACKGROUND

Customer service may be provided to customers in various contexts, suchas before, during, and after purchases of goods and services. Customerservice provided to customers may be provided in a manual way (e.g., byhumans) or in an automated way (e.g., using kiosks, applications,artificial intelligence, and the like).

BRIEF DESCRIPTION OF THE DRAWINGS

The teachings of the present disclosure can be readily understood byconsidering the following detailed description in conjunction with theaccompanying drawings, in which:

FIG. 1 illustrates an example system configured to support automation ofcustomer service based on analysis of customer service conversationsbetween customers and customer service agents;

FIG. 2 illustrates a flowchart of an example method for supportingautomation of customer service based on analysis of customer serviceconversations between customers and customer service agents;

FIG. 3 illustrates a flowchart of an example method for determining aquantitative ranking of conversation-level topics from customer serviceconversations between customers and customer service agents;

FIG. 4 illustrates a flowchart of an example method for determining aquantitative ranking of utterance-level topics from customer serviceconversations between customers and customer service agents;

FIG. 5 illustrates a flowchart of an example method for initiating acustomer service automation action based on analysis of a set ofcustomer service conversation records; and

FIG. 6 illustrates a high level block diagram of a computing systemspecifically programmed to perform the steps, functions, blocks and/oroperations described herein.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe figures.

DETAILED DESCRIPTION

The present disclosure describes various methods, computer-readablemedia, and apparatuses for supporting automation of customer service.The support for automation of customer service may be based on analysisof customer service conversations between customers and customer serviceagents of a customer service center. The support for automation ofcustomer service may be based on analysis of customer serviceconversations between customers and customer service agents to identifya set of topics (e.g., conversation-level topics, utterance-leveltopics, and so forth) from the customer service conversations betweenthe customers and the customer service agents. The support forautomation of customer service may be based on performing analysis ofcustomer service conversations between customers and customer serviceagents to identify a set of topics from the customer serviceconversations between the customers and the customer service agents andsupporting automation of customer service based on the set of topicsfrom the customer service conversations between customers and customerservice agents. The support for automation of customer service may bebased on performing analysis of customer service conversations betweencustomers and customer service agents to determine a quantitativeranking of a set of topics from the customer service conversationsbetween the customers and the customer service agents and supportingautomation of customer service based on the quantitative ranking of theset of topics from the customer service conversations between customersand customer service agents. The support for automation of customerservice may include obtaining a set of customer conversation records fora set of customer service conversations between customers and customerservice agents, identifying, based on processing of the set of customerservice conversation records, a set of topics of the customer serviceconversations (e.g., conversation-level topics, utterance-level topics,and so forth), determining a quantitative ranking of the topics in theset of topics of the customer service conversations, and initiating amanagement action based on the quantitative ranking of the topics in theset of topics of the customer service conversations. The support forautomation of customer service may be provided based on analysis of thecustomer service conversations at the conversation level, therebyenabling identification and quantitative ranking of conversation-leveltopics and support for various management actions which may be providedbased on the identification and quantitative ranking of theconversation-level topics (e.g., identification of topics for whichchatbots are to be created, identification of problems, monitoring andtracking of trends, and so forth). The support for automation ofcustomer service may be provided based on analysis of the customerservice conversations at the utterance level (e.g., for a givenconversation-level topic), thereby enabling identification andquantitative ranking of utterance-level topics and support for variousmanagement actions which may be provided based on the identification andquantitative ranking of the utterance-level topics (e.g., design ofspecific features to be provided by chatbots, monitoring and tracking oftrends, and so forth). The support for automation of customer servicebased on analysis of the customer service conversations may be performedbased on various capabilities, algorithms (e.g., clustering algorithms,topic extraction algorithms, and the like), models (e.g., generalstatistical models, Latent Dirichlet Allocation (LDA) models, and thelike), evaluation criteria, and so forth. It will be appreciated that atleast some such functions described in conjunction with support forautomation of customer service also may be provided outside of thecontext of support for automation of customer service (e.g., supportingidentification of repetitive topics from the customer serviceconversations between customers and customer service agents for use inother ways, supporting identification and quantitative ranking ofrepetitive topics from the customer service conversations betweencustomers and customer service agents for use in other ways, and soforth).

In one example, a method is provided. The method includes obtaining, bya processing system including at least one processor, a set of customerservice conversation records, wherein each of the customer serviceconversation records includes a respective set of utterances. The methodincludes dividing, by the processing system, the set of customer serviceconversation records into a set of clusters, wherein each of theclusters represents a respective group of the customer serviceconversation records having a respective conversation-level topic incommon. The method includes determining, by the processing system fromthe set of clusters, a set of primary conversation-level topics. Themethod includes obtaining, by the processing system for at least one ofthe primary conversation-level topics associated with at least one ofthe clusters, a set of keywords, wherein the set of keywords is obtainedbased on the at least one of the primary conversation-level topics,wherein the set of keywords is obtained from a source different than theset of clusters. The method includes determining, by the processingsystem from the utterances of the at least one of the clusters of the atleast one of the primary conversation-level topics, a set ofutterance-level topics associated with the at least one of the primaryconversation-level topics, wherein the set of utterance-level topics isdetermined based on the set of keywords. The method includes initiating,by the processing system based on the at least one of the primaryconversation-level topics and the set of utterance-level topics, acustomer service automation action.

In one example, a non-transitory computer-readable storage medium storesinstructions which, when executed by a processing system, cause theprocessing system to perform operations. The operations includeobtaining a set of customer service conversation records, wherein eachof the customer service conversation records includes a respective setof utterances. The operations include dividing the set of customerservice conversation records into a set of clusters, wherein each of theclusters represents a respective group of the customer serviceconversation records having a respective conversation-level topic incommon. The operations include determining, from the set of clusters, aset of primary conversation-level topics. The operations includeobtaining, for at least one of the primary conversation-level topicsassociated with at least one of the clusters, a set of keywords, whereinthe set of keywords is obtained based on the at least one of the primaryconversation-level topics, wherein the set of keywords is obtained froma source different than the set of clusters. The operations includedetermining, from the utterances of the at least one of the clusters ofthe at least one of the primary conversation-level topics, a set ofutterance-level topics associated with the at least one of the primaryconversation-level topics, wherein the set of utterance-level topics isdetermined based on the set of keywords. The operations includeinitiating, based on the at least one of the primary conversation-leveltopics and the set of utterance-level topics, a customer serviceautomation action.

In one example, an apparatus includes a processing system including atleast one processor and a computer-readable medium storing instructionswhich, when executed by the processing system, cause the processingsystem to perform operations. The operations include obtaining a set ofcustomer service conversation records, wherein each of the customerservice conversation records includes a respective set of utterances.The operations include dividing the set of customer service conversationrecords into a set of clusters, wherein each of the clusters representsa respective group of the customer service conversation records having arespective conversation-level topic in common. The operations includedetermining, from the set of clusters, a set of primaryconversation-level topics. The operations include obtaining, for atleast one of the primary conversation-level topics associated with atleast one of the clusters, a set of keywords, wherein the set ofkeywords is obtained based on the at least one of the primaryconversation-level topics, wherein the set of keywords is obtained froma source different than the set of clusters. The operations includedetermining, from the utterances of the at least one of the clusters ofthe at least one of the primary conversation-level topics, a set ofutterance-level topics associated with the at least one of the primaryconversation-level topics, wherein the set of utterance-level topics isdetermined based on the set of keywords. The operations includeinitiating, based on the at least one of the primary conversation-leveltopics and the set of utterance-level topics, a customer serviceautomation action.

These and various other aspects of the present disclosure are discussedin greater detail below in connection with the examples of FIGS. 1-6 .

FIG. 1 illustrates an example system configured to support automation ofcustomer service based on analysis of customer service conversationsbetween customers and customer service agents.

The system 100 may be configured to support customer service for a setof customers 101-1 to 101-N (collectively, customers 101). The customers101 may be customers of one or more entities (e.g., companies,organizations, and the like) which may provide customer service for thecustomers 101 and, thus, may have relationships with the entities whichmay provide customer service for the customers 101. The customers 101may receive customer service within various contexts, such as within thecontexts of customer support (e.g., product support, service support,and the like), electronic commerce (e-commerce), entertainment,education, news, finance, healthcare, travel, and so forth. It will beappreciated that, although primarily presented herein with respect tosupporting automation of customer service for customers 101 withinspecific customer service contexts (e.g., primarily within the contextof technical support), automation of customer service for customers 101may be provided within various other customer service contexts.

The system 100 includes a set of customer devices 110-1 to 110-N(collectively, customer devices 110) of the set of customers 101-1 to101-N, respectively, a communication network 120, and a customer servicecenter 130.

The customers 101 may use the customer devices 110 to interact with thecustomer service center 130. It will be appreciated, as discussedfurther below, that the customer devices 110 may or may not be relatedto the relationship between the customers 101 and the entity for whichcustomer service is provided by the customer service center 130.

In one example, the customer devices 110 of the customers 101 may berelated to the relationship between the customers 101 and the entity forwhich customer service is provided by the customer service center 130.For example, the customers 101 may be customers of an end user devicecompany from which the customers 101 may buy or rent the customerdevices 110, the customers 101 may be customers of a telecommunicationsservice provider from which the customers 101 may obtaintelecommunications services using the customer devices 110, and soforth.

In one example, the customer devices 110 of the customers 101 may beunrelated to the relationship between the customers 101 and the entityfor which customer service is provided by the customer service center130. For example, the customer devices 110 may be devices via which thecustomers 101 are able to interact with the company within the contextof receiving customer service from the company, even though the customerservice may be unrelated to the customer device 110. For example, acustomer 101 may call customer service via a customer device 110regarding a cable service subscribed to by the customer 101, initiate achat session with customer service via a customer device 110 regarding aproduct purchased by the customer 101, and so forth.

The customer devices 110 of the customers 101 may include any devicesvia which the customers 101 may interact with the customer servicecenter 130 for obtaining customer service from the customer servicecenter 130. For example, the customer devices 110 may be desktopcomputers, laptop computers, tablet computers, smartphones, wearabledevices, smart televisions, gaming systems, and so forth.

The communication network 120 may include one or more types ofcommunication networks which may support communications between thecustomers 101 using the customer devices 110 and the customer servicecenter 130. For example, the communication network 120 may include atraditional circuit switched network (e.g., a public switched telephonenetwork (PSTN)). For example, the communication network 120 may includea packet network, such as an Internet Protocol (IP) network (e.g., aVoice over IP (VoIP) network, a Service over IP (SoIP) network, an IPMultimedia Subsystem (IMS) network, and the like), an asynchronoustransfer mode (ATM) network, a wireless network (e.g., a cellularnetwork such as a 2G network, a 3G network, a 4G network, a long termevolution (LTE) network, a 5G network, and the like), and so forth. Itwill be appreciated that the communication network 120 may include oneor more access networks, one or more core networks, and so forth.

The communication network 120 includes communication systems 121configured to support communications between the customers 101 usingcustomer devices 110 and the customer service center 130. For example,communication systems 121 may include network elements (e.g., accessdevices (e.g., base transceiver stations (BTSs), WiFi access points(APs), and so forth), routers, switches, hubs, and so forth), networkcontrol devices, virtualization infrastructure configured to supportnetwork function virtualization (NFV) based on virtualized networkfunctions (VNFs), servers configured to support various applications(e.g., policy servers, mobility servers, web servers, content servers,and the like), management systems, and so forth.

The customer service center 130 may be configured to provide customerservice for the customers 101. The customer service center 130 may beconfigured to provide customer service for the customers 101 based onvarious customer service support functions supported by the customerservice center 130. For example, customer service support functions mayinclude customer communication functions for supporting communicationswith customers 101, customer information management functions formanaging (e.g., collecting, maintaining, accessing, and the like)customer information for supporting customer service for the customers101 (e.g., account information, preference information, and the like),customer service functions (e.g., customer care functions, customerexperience management functions, and the like), and so forth. It will beappreciated that the customer service support functions supported by thecustomer service center 130 may vary across entities (e.g., based onentity types of the entities, customer service contexts of the entities,and the like), may vary across customers 101, and so forth. The customerservice center 130 may be configured to support customer service supportfunctions based on various communication types which may be used by thecustomers 101 (e.g., text-based communications, audio-basedcommunications, video-based communications, and so forth). The customerservice center 130 may be configured to support various other functionsfor providing customer service for the customers 101.

For example, where the entity is a telecommunications service providerand the customers 101 use telecommunications services of thetelecommunications service provider, the customer service supportfunctions supported by the customer service center 130 may includetelecommunications service technical support functions. For example,telecommunications service technical support functions may includetechnical support for telecommunications services (e.g., problemsaccessing or using the service), technical support for devices via whichcustomers 101 access telecommunications services (e.g., customer devices110 or other devices), and so forth.

For example, where the entity is an end user device provider and thecustomers 101 purchase or rent end user devices (e.g., desktopcomputers, laptop computers, tablet computers, smartphones, and thelike), the customer service support functions supported by the customerservice center 130 may include end user device technical supportfunctions. For example, end user device technical support functions mayinclude technical support for issues with end user devices (e.g.,problems with network connectivity, problems with software updates, andso forth).

For example, where the entity is an e-commerce company that sellsproducts online, the customer service support functions supported by thecustomer service center 130 may include product support functions. Forexample, product support functions may include support for findingproducts of interest, support for researching products of interest,support for ordering products, support for tracking delivery ofproducts, support for issues with products, support for returning orexchanging products, and so forth.

It will be appreciated that the foregoing examples represent merely afew of the types of entities for which the customer service center 130may provide customer service to the customers 101.

The customer service center 130 includes customer service agents 131which may provide customer service to the customers 101 based on variouscustomer service support functions supported by the customer servicecenter 130. The customer service agents 131 may provide customer serviceto the customers 101 via customer service conversations which may beconducted between the customer service agents 131 and the customers 101for purposes of providing customer service to the customers 101 (e.g.,audio-based conversations via voice calls, text-based conversations viatext-based chat interfaces, and the like). The customer service agents131 may include a set of human customer service agents 131-H which mayinteract with customers 101 using customer service agent terminals 132and a set of automated customer service agents 131-A. The customerservice agents 131 may be able to communicate with the customers 101using various types of communications (e.g., audio, text, video, and soforth).

The human customer service agents 131-H, as indicated above, mayinteract with the customers 101 to provide customer service functionsfor the customers 101. The human customer service agents 131-H interactwith the customers 101 using the customer service agent terminals 132.The customer service agent terminals 132 may include any devices andcapabilities which may be used by the human customer service agents131-H to interact with customers 101. For example, for text-basedinteraction between the human customer service agents 131-H and thecustomers 101, the customer service agent terminals 132 may beconfigured to enable the human customer service agents 131-H to acceptchat sessions initiated by customers 101, view chat messages sent bycustomers 101, send chat messages to customers 101, add other customerservice agents 131 to chat sessions with customers 101, and so forth.For example, for voice-based interaction between the human customerservice agents 131-H and the customers 101, the customer service agentterminals 132 may be configured to enable the human customer serviceagents 131-H to view calls in a queue of pending calls to be handled,select calls to answer calls of customers 101, interact with customers101 verbally, terminate calls with customers 101, add other customerservice agents 131 to calls, transfer calls of customers 101 to othercustomer service agents 131, and so forth. For example, the customerservice agent terminals 132 may include workstations, computers,smartphones, applications (e.g., running on physical devices orresources, virtual resources (e.g., virtual machines (VMs), virtualcontainers (VCs), and the like), and the like), and so forth. It will beappreciated that references herein to functions performed by a customerservice agent 131, in the case of a human customer service agent 131-H,may be performed by one or more elements on behalf of the human customerservice agent 131-H (e.g., by the customer service agent terminal 132 orother devices or elements). The human customer service agents 131-H mayprovide various other customer service functions for the customers 101.

The automated customer service agents 131-A, as indicated above, mayinteract with the customers 101 to provide customer service functionsfor the customers 101. The automated customer service agents 131-A maybe automated in various ways based on various automation technologies.The automated customer service agents 131-A may be based on artificialintelligence (Al). The automated customer service agents 131-A may bechatbots. For example, the automated customer service agents 131-A mayinclude chatbots configured to communicate with customers 101 usingaudio, text, and so forth. For example, the automated customer serviceagents 131-A may include chatbots which may have audio-based interfaces,text-based interfaces, video-based interfaces, and so forth. Theautomated customer service agents 131-A may be provided using physicaldevices or resources, virtual resources (e.g., VMs, VCs, and the like),and so forth. It will be appreciated that references herein to functionsperformed by a customer service agent 131, in the case of an automatedcustomer service agent 131-A, may be performed by the automated customerservice agent 131-A, by one or more elements on behalf of the automatedcustomer service agent 131-A, and so forth. The automated customerservice agents 131-A may provide various other customer servicefunctions for the customers 101.

The customer service center 130 includes a customer service automationsupport system 135 that is configured to support automation of customerservice provided to the customers 101 by the customer service agents 131of the customer service center 130. The customer service automationsupport system 135 may be configured to support automation of customerservice based on analysis of customer service conversation records forcustomer service conversations between customers 101 and customerservice agents 131. The customer service automation support system 135may be configured to support automation of customer service by obtainingcustomer service conversation records for customer service conversationsbetween customers 101 and customer service agents 131, analyzing thecustomer service conversation records for the customer serviceconversations between the customers 101 and the customer service agents131 for determining a quantitative ranking of topics discussed in thecustomer service conversations between the customers 101 and thecustomer service agents 131, and initiating a management action relatedto the customer service provided to the customers 101 by the customerservice agents 131 based on the quantitative ranking of topics discussedin the customer service conversations between the customers 101 and thecustomer service agents 131. An example method for use by the customerservice automation support system 135 to support automation of customerservice based on analysis of customer service conversation records forcustomer service conversations between customers 101 and customerservice agents 131 is presented in FIG. 2 . It will be appreciated thatthe customer service automation support system 135 may be configured tosupport various other functions for supporting automation of customerservice based on analysis of customer service conversation records forcustomer service conversations between customers 101 and customerservice agents 131.

The customer service automation support system 135 obtains customerservice conversation records for customer service conversations betweencustomers 101 and customer service agents 131. The customer serviceconversation records for customer service conversations betweencustomers 101 and customer service agents 131 may be captured by the oneor more elements of the customer service center 130 (e.g., recording andstoring audio conversations, capturing and storing text-basedconversations, and so forth) such that the customer service conversationrecords are available to customer service automation support system 135.The customer service conversation records may include audio-basedrecords (e.g., for voice-based conversations between the customers 101and the customer service agents 131), text-based records (e.g., fortext-based conversations between the customers 101 and the customerservice agents 131), and so forth. The customer service conversationrecords may include conversations between customers 101 and customerservice agents 131 that may include non-topic-related utterances (e.g.,greetings, chitchat, and so forth), topic-related utterances related tovarious topics which may be discussed within the context of the customerservice agents 131 providing customer service to the customers 101, andso forth. For example, non-topic-related utterances may includestatements such as ‘Hello, how can I help you today,’ ‘Please hold forone moment,’ ‘Thank you for your patience,’ and the like. For example,topic-related utterances may include phrases related to customer servicetopics (e.g., ‘Please give me a moment to access your account,’ ‘I amhaving trouble accessing the Internet,’ ‘I am having a hard timelocating a product,’ ‘I would like to pay my outstanding balance,’ andthe like), words related to customer service topics (e.g., ‘account,’‘modem,’ ‘router,’ ‘phone,’ ‘product,’ ‘payment,’ and the like), and soforth. As discussed further below, the topics which may be discussedwithin the customer service conversations between the customers 101 andthe customer service agents 131 may include conversation-level topics,utterance-level topics, and so forth. It will be appreciated that thecustomer service conversations between customers 101 and customerservice agents 131 may include various other utterances andconversations related to various other topics which may be discussed ina customer service context.

The customer service automation support system 135 analyzes the customerservice conversation records for the customer service conversationsbetween the customers 101 and the customer service agents 131 to extracttopics from the customer service conversations between the customers 101and the customer service agents 131, analyze topics extracted from thecustomer service conversations between the customers 101 and thecustomer service agents 131, determine quantitative rankings of topicsextracted from the customer service conversations between the customers101 and the customer service agents 131, and so forth. As discussedfurther below, the analysis of the customer service conversation recordsfor the customer service conversations between the customers 101 and thecustomer service agents 131 that is performed by the customer serviceautomation support system 135 may be different for conversation-leveltopics and utterance-level topics.

In one example, customer service conversations between the customers 101and the customer service agents 131 may include conversation-leveltopics. A conversation-level topic of a customer service conversationmay be a topic that is identified based on analysis of a set ofutterances in a customer service conversation record of the customerservice conversation. The quantitative ranking of the conversation-leveltopics from the customer service conversations between the customers 101and the customer service agents 131 may be determined based ongeneration of conversational documents (e.g., including topic-relatedutterances from the customer service conversations between the customers101 and the customer service agents 131 and excluding non-topic-relatedutterances from the customer service conversations between the customers101 and the customer service agents 131), processing of theconversational documents to provide conversational documentrepresentations of the conversational documents (e.g., based on use ofneural networks, clustering of the conversational documentrepresentations to form clusters representing groups of customer serviceconversations having one or more topics in common (e.g., based on aclustering algorithm), analysis of the clusters to identifyconversation-level topics of the clusters (e.g., based on analysis ofthe sets of conversational document representations in the clustersusing one or more topic extraction algorithms)), and determination ofthe quantitative ranking of the conversation-level topics associatedwith the clusters (e.g., based on cluster sizes of the clusters or otherquantitative ranking criteria). The quantitative ranking of theconversation-level topics from the customer service conversations may bedetermined based on dividing of the set of customer service conversationrecords into a set of clusters where each cluster in the set of clustersrepresents a respective group of the customer service conversationrecords having a respective conversation-level topic in common,determination of a set of primary conversation-level topics for the setof customer service conversation records from the set of clusters (e.g.,one primary conversation-level topic per cluster, fewer or more than oneprimary conversation-level topic per cluster, and so forth), anddetermination of the quantitative ranking of the primaryconversation-level topics for the set of customer service conversationrecords. It is noted that the primary conversation-level topic(s)determined from a given cluster may be the same as or different than theconversation-level topic in common which caused the grouping of thatsubset of the customer service conversation records to form the givencluster. The quantitative ranking of the conversation-level topics fromthe customer service conversations between the customers 101 and thecustomer service agents 131 may be determined based on various othertypes of capabilities, analysis, algorithms, models, criteria, and soforth. In one example, the quantitative ranking of theconversation-level topics from the customer service conversationsbetween the customers 101 and the customer service agents 131 may bedetermined as presented with respect to FIG. 3 .

In one example, customer service conversations between the customers 101and the customer service agents 131 may include utterance-level topics.An utterance-level topic of a customer service conversation may be atopic that is identified based on analysis of a set of words or phrasesin an utterance in a customer service conversation record of a customerservice conversation. The quantitative ranking of the utterance-leveltopics from the customer service conversations between the customers 101and the customer service agents 131 may be determined on aper-conversation-level topic basis for one or more conversation-leveltopics identified based on processing of the customer serviceconversations between the customers 101 and the customer service agents131 (e.g., based on analysis of the conversational documentrepresentations, or customer service conversations represented by theconversational document representations, of the cluster of theconversation-level topic for which the utterance-level topics are beingidentified and quantitatively ranked). The quantitative ranking of theutterance-level topics from the customer service conversations betweenthe customers 101 and the customer service agents 131 may be determinedbased on obtaining, from a cluster of customer service conversationsassociated with a conversation-level topic, a set of utterancesassociated with the conversation-level topic, obtaining a set ofkeywords associated with the conversation-level topic (e.g., abusiness-specific vocabulary related to the conversation-level topic)based on the conversation-level topic and from a source different thanthe respective set of utterances of the respective cluster of customerservice conversations associated with the conversation-level topic,extracting a set of phrases (e.g., a set of business-specific phrases)from the set of utterances associated with the conversation-level topicbased on the set of keywords, extracting the set of utterance-leveltopics from the set of utterances associated with the conversation-leveltopic based on the set of phrases, and determining the quantitativeranking of the utterance-level topics in the set of utterance-leveltopics. The quantitative ranking of the utterance-level topics from thecustomer service conversations between the customers 101 and thecustomer service agents 131 may be determined based on various othertypes of analysis, algorithms, models, criteria, and so forth. In oneexample, the quantitative ranking of the utterance-level topics from thecustomer service conversations between the customers 101 and thecustomer service agents 131 may be determined as presented with respectto FIG. 4 .

It will be appreciated that the types of utterances and topics which maybe included within, and identified from, a customer service conversationrecord for a customer service conversation between one of the customers101 and one of the customer service agents 131 may be further understoodwith respect to a specific example. For example, if a customer 101 callsin to resolve a problem with his or her credit card billing information,the customer 101 and the customer service agent 131 may first greet eachother (a non-topic-related utterance), the customer 101 may indicate aneed to correct credit card billing information, the customer serviceagent 131 may verify the account information of the customer 101, thecustomer service agent 131 may verify the customer name of the customer101, the customer service agent 131 may verify the credit cardinformation of the customer 101, and the customer service agent 131 maythen help the customer 101 by correcting the credit card information ofthe customer 101. Here, the main topic of the customer serviceconversation record, which may be a candidate to be a conversation-leveltopic (e.g., based on clustering of this customer service conversationwith other customer service conversations also determined to be relatedto this main topic), may be “correct credit card billing information” orsomething similar. Here, the customer service conversation record alsoincludes other topics, some of which are directly related to the maintopic (and, thus, which may be candidates to represent theconversation-level topic and which also may be candidates to beutterance-level topics associated with the conversation-level topic) andsome of which are unrelated to the main topic (and, thus, which may notbe candidates to be utterance-level topics associated with the maintopic). For example, at least portions of the utterances related to theneed to correct credit card billing information, verification of thename of the customer 101, verification of the credit card information ofthe customer 101, and correction of the credit card information of thecustomer 101 may be considered to be directly related to the main topic.For example, at least a portion of the utterance related to verificationof the account information of the customer 101 may be considered to beunrelated to the main topic (e.g., since this type of verification maybe performed for all interactions between customers and customer serviceagents 131 such that it may not be considered to be useful inidentifying a conversation-level topic and may or may not be consideredto be useful in identifying utterance-level topics associated with theconversation-level topic). It will be appreciated that, since the aboveexample is merely a single customer service conversation record, andidentification and quantitative ranking of conversation-level topics maybe performed based on clustering of multiple customer serviceconversations of multiple customer service conversation records, thedetermination as to which of the utterances in the customer serviceconversation record are related to candidate topics of interest andwhich are unrelated to candidate topics of interest may depend onprocessing of the customer service conversation record in combinationwith other customer service conversation records related to customerservice conversations which are clustered based on identification ofcommon topics.

The customer service automation support system 135, based on analysis ofcustomer service conversation records for customer service conversationsbetween customers 101 and customer service agents 131, initiates amanagement action related to the customer service provided to thecustomers 101 by the customer service agents 131. As discussed furtherbelow, the management action that is initiated by the customer serviceautomation support system 135 may be different for conversation-leveltopics and utterance-level topics.

In one example, the management action may include providing aquantitative ranking of topics from the customer service conversationsbetween the customers 101 and the customer service agents 131 (e.g., aquantitative ranking of conversation-level topics, a quantitativeranking of utterance-level topics, and so forth). The providing of thequantitative ranking of topics from the customer service conversationsbetween the customers 101 and the customer service agents 131 mayinclude storing the quantitative ranking of the topics from the customerservice conversations (e.g., such that the quantitative ranking may beaccessed by various personnel, such as customer service centerautomation personnel, for various purposes), displaying the quantitativeranking of the topics from the customer service conversations (e.g., tovarious personnel, such as customer service center automation personnel,such that the quantitative ranking may be used for various purposes),sending the quantitative ranking of the topics from the customer serviceconversations to one or more devices (e.g., one or more managementsystems, one or more end user devices of various personnel such ascustomer service center automation personnel, and the like), and thelike.

In one example, the management action may include generating contentbased on analysis of customer service conversation records for customerservice conversations between customers 101 and customer service agents131. The content may be generated based on a quantitative ranking oftopics from the customer service conversations between the customers 101and the customer service agents 131. The content that is generated mayinclude one or more reports presenting results of the analysis of thecustomer service conversation records (e.g., presenting a quantitativeranking of the topics from the customer service conversations forenabling personnel to visualize the quantitative ranking of the topicsfrom the customer service conversations), one or more recommendationsdetermined based on the analysis of customer service conversationrecords (e.g., a recommendation to create a chatbot to automate aparticular conversation-level topic, a recommendation to address aproblem, a recommendation regarding particular functions that should besupported by a chatbot to address particular utterance-level topics, andthe like), one or more software programs automatically created based onthe analysis of customer service conversation records and which may beused to automate one or more customer service functions (e.g., one ormore chatbot programs, software modules, and the like), one or morealerts generated based on the analysis of customer service conversationrecords (e.g., alerts which may identify problems or potential problemswith products or services, alerts which may identify problems orpotential problems with aspects of customer service, alerts indicativeof major problems encountered, alerts indicative of newly identifiedcustomer issues, alerts indicative of changes in issue trends, and thelike), and so forth.

In one example, the management action may include supporting a customerservice center automation initiative. The customer service centerautomation initiative may be supported by providing a recommendation fora customer service center automation initiative, automaticallyinitiating a customer service center automation initiative, and soforth.

In one example, where the topics are conversation-level topics, thecustomer service center automation initiative may include providing achatbot. The chatbot may be provided for one of the conversation-leveltopics based on a quantitative ranking of the conversation-level topicsfrom the customer service conversations between the customers 101 andthe customer service agents 131 (e.g., the conversation-level topichaving the highest ranking). Here, the quantitative ranking of theconversation-level topics from the customer service conversationsbetween the customers 101 and the customer service agents 131 provides alist of repetitive topics that are discussed relatively often duringconversations between the customers 101 and the customer service agents131 and, thus, which are good candidates for automation based ondevelopment of new chatbots that can automate handling of those topicsand can therefore provide more efficient and cost-effective customersupport for the customers 101. In other words, this enablesdetermination of which types of customer service conversation topicsshould have the highest automation priority, thereby supportingprioritization of chatbot development for supporting customer servicefor the customers 101. For example, if the most commonly discussedconversation-level topic identified in the customer serviceconversations between the customers 101 and the customer service agents131 is the topic of troubleshooting problems with a wireless router,then a chatbot that is able to automate this function for customers 101may be developed and deployed within customer service center 130 toimprove the automation of customer service for the customers 101.

In one example, where the topics are utterance-level topics, thecustomer service center automation initiative may include supportingdesign of a chatbot. The design of the chatbot may be based on one ormore of the utterance-level topics in a quantitative ranking of theutterance-level topics from the customer service conversations betweenthe customers 101 and the customer service agents 131 (e.g., one or moreof the utterance-level topics having the highest ranking(s)). Here, thequantitative ranking of the utterance-level topics from the customerservice conversations between the customers 101 and the customer serviceagents 131 provides a list of repetitive topics that are utteredrelatively often during customer service conversations between thecustomers 101 and the customer service agents 131 and, thus, which maybe used to support the design and development of a chatbot for theconversation-level topic with which the utterance-level topics areassociated. It is noted that, while a conversation-level topic may beused to decide on a type of chatbot which may be developed to automatehandling of that conversation-level topic, the conversation-level topicmay not necessarily provide an indication of specific topics to behandled for the various utterances which might be exchanged between thecustomers 101 and the customer service agents 131 within the context ofthat conversation-level topic; however, the identification and rankingof the utterance-level topics within that conversation-level topic maybe used as a basis (e.g., roadmap) for designing the chatbot for thatconversation-level topic (e.g., designing the chatbot to handle specificones of the utterance-level topics in order to provide overall supportfor the conversation-level topic). In other words, this enablesdetermination of design aspects of the conversational dialog of achatbot to support specific types of functions for the conversationaltopic for which the chatbot is provided, thereby supportingprioritization of chatbot development for supporting customer servicefor the customers 101. For example, if the most commonly discussedutterance-level topics identified in the customer service conversationsbetween the customers 101 and the customer service agents 131 includeutterances of “port” and “connection” within the context of theconversation-level topic of troubleshooting problems with a wirelessrouter, then the design of the chatbot to automate this function forcustomers 101 (where, as indicated above, the chatbot may be developedand deployed within customer service center 130 to improve theautomation of customer service for the customers 101) may includedesigning the chatbot to interact with the customer 101 to verify theport of the wireless router and to interact with the customer 101 toverify the connections of the wireless router, respectively.

In one example, the management action may include initiating a customerservice automation action. The customer service automation action mayinclude providing a recommendation for a chatbot to be created orautomatically creating the chatbot (e.g., a chatbot configured to handlea particular conversation-level topic where the particularconversation-level topic is selected based on a quantitative ranking ofconversation-level topics), providing a recommendation for a chatbot tobe created and for a design for the chatbot or automatically creating achatbot having a particular design (e.g., a chatbot configured to handlea particular conversation-level topic where the particularconversation-level topic is selected based on a quantitative ranking ofconversation-level topics and configured to support tasks related toparticular utterance-level topics where such utterance-level topics maybe selected based on a quantitative ranking of utterance-level topicsassociated with the particular conversation-level topic), and so forth.It will be appreciated that automated creation of a chatbot or othersoftware may be based on various artificial intelligence (AI) techniquesand/or use of other techniques or capabilities configured to supportautomated software creation.

In one example, the management action may include identification ofproblems associated with the basis for which customer service isprovided (e.g., problems associated with products where the entity forwhich customer service is provided is a product provider, problemsassociated with services where the entity for which customer service isprovided is a service provider, and so forth). This management actionmay be initiated where the topics are conversation-level topics. Theconversation-level topics determined to be discussed most frequently inthe customer service conversations between the customers 101 and thecustomer service agents 131 may be indicative of problems. For example,for a smartphone provider, where the most frequently discussed topicrelates to problems with battery life of a particular model of thesmartphone, the inclusion of this topic at the top of the quantitativerankings of the conversation-level topics from the customer serviceconversations between the customers 101 and the customer service agents131 may trigger identification of battery life limitations as an urgentproblem that should be addressed by the smartphone provider. Forexample, for an e-commerce company, where the most frequently discussedtopic relates to problems associated with a shopping cart page, theinclusion of this topic at the top of the quantitative rankings of theconversation-level topics from the customer service conversationsbetween the customers 101 and the customer service agents 131 maytrigger identification of the design of the shopping cart page as anurgent problem that should be addressed by the e-commerce company. Itwill be appreciated that problems may be identified in various ways,various other problems may be identified, and so forth.

In one example, the management action may include detection andmonitoring of trends associated with the basis for which customerservice is provided (e.g., trends associated with products where theentity for which customer service is provided is a product provider,trends associated with services where the entity for which customerservice is provided is a service provider, and so forth). Thismanagement action may be initiated where the topics areconversation-level topics. The conversation-level topics determined tobe discussed most frequently in the customer service conversationsbetween the customers 101 and the customer service agents 131 may changeover time and the tracking of the conversation-level topics over timemay enable detection and monitoring of trends in the conversation-leveltopics over time. For example, for a smartphone provider, where batterylife limitations were not previously identified as a relatively commontopic of discussion in customer service conversations between thecustomers 101 and the customer service agents 131, but then begin toappear at or near the top of the quantitative rankings of theconversation-level topics after release of a new model of smartphone,the smartphone provider may identify this trend change as beingindicative of a problem with the battery of the new smartphone model.For example, for an e-commerce company, where a shopping cart page hasbeen identified as an urgent problem that should be addressed by thee-commerce company, the e-commerce company may redesign the shoppingcart page and continue to monitor the conversation-level topics overtime to determine whether the redesign of the shopping cart page hassolved the problem (e.g., if the shopping cart page continues to be atthe top of the quantitative rankings of the conversation-level topicsthen the e-commerce company can determine that the redesign probably didnot solve the problem, but if the shopping cart page drops out of thetop of the quantitative rankings of the conversation-level topics thenthe e-commerce company can determine that the redesign probably solvedthe problem). It will be appreciated that trends may be detected andtracked in various ways, various other trends may be detected andtracked, and so forth.

It will be appreciated that various other management actions may beinitiated based on analysis of customer service conversation records ofthe customer service conversations between the customers 101 and thecustomer service agents 131.

It will be appreciated that various combinations of such managementactions may be initiated. It will be appreciated that variouscombinations of such management actions may be initiated in various ways(e.g., serially, in parallel, and so forth).

The customer service automation support system 135, as indicated above,may be configured to process customer service conversation records tosupport various customer service automation functions for supportingautomation of customer service for customers 101. This is illustrated inFIG. 1 as customer service conversation records 136 available to thecustomer service automation support system 135 and which may beprocessed by the customer service automation support system 135, basedon customer service automation inputs 137, to produce customer serviceautomation outputs 138 maintained by the customer service automationsupport system 135.

The customer service automation inputs 137 may include any inputs whichmay be used by the customer service automation support system 135 forprocessing the customer service conversation records 136 to produce thecustomer service automation outputs 138. For example, the customerservice automation inputs 137 may include topic extraction algorithms,topic extraction models, neural feature extraction algorithms, set ofkeywords associated with conversation-level topics (e.g., different setsof keywords for different conversation-level topics), sets of keywordsassociated with utterance-level topics (e.g., different sets of keywordsfor different utterance-level topics), and so forth.

The customer service automation outputs 138 may include any outputswhich may be produced by the customer service automation support system135 based on processing of the customer service conversation records136. For example, the customer service automation outputs 138 mayinclude lists of topics determined by the customer service automationsupport system 135 (e.g., conversation-level topics (e.g., topics incommon, primary topics, and so forth), utterance-level topics, and soforth), quantitative rankings of topics determined by the customerservice automation support system 135, content generated by the customerservice automation support system 135 based on list of topics and/orquantitative rankings of topics, outputs used for management actionsinitiated by the customer service automation support system 135, outputsgenerated as a result of management actions initiated by the customerservice automation support system 135, and so forth.

It will be appreciated that, although omitted for purposes of clarity,the customer service automation support system 135 may be configured tomaintain or control various other types of information (e.g., variousother input information in addition to the customer service conversationrecords 136, various intermediate information which may be producedbased on processing of the customer service conversation records 136,various other types of outputs in addition to the customer serviceautomation outputs 138, various programs which may be executed bycustomer service automation support system 135 to provide variouscustomer service automation support functions (e.g., various programswhich may be configured to support processing of the customer serviceconversation records 136 for identification and quantitative ranking oftopics, various programs which may be configured to support managementactions which may be initiated based on processing of the customerservice conversation records 136 for identification and quantitativeranking of topics, various programs which may be configured to supportgeneration and/or analysis of customer service automation outputs 138,and the like), and so forth.

The customer service automation support system 135 may be configured tosupport various conversation monitoring functions. The customer serviceautomation support system 135 may be configured to support continuousmonitoring of conversational topics so that if there is any change inthe topic distribution or any new topic or customer issues emerge, oneor more management actions may be initiated (e.g., providing anotification to an automation team or a customer service team,automatically generating software which may be executed to address theemerging topic or issue, and so forth). The customer service automationsupport system 135 may be configured to support continuous monitoring ofconversational topics by determining, for a new customer serviceconversation record that is received (e.g., obtained as text or obtainedas audio and translated to text through a “speech-to-text” module),whether the new customer service conversation record belongs to anyexisting clusters of customer service conversation records. The mannerin which this determination is performed may depend on the manner inwhich the clustering of the customer service conversation records isperformed. For example, where the clustering of the customer serviceconversation records is performed using K-means clustering as theclustering algorithm, then, after training, the clustering algorithmgenerates a set of K conversational cluster centers of the clusters anddetermines respective distances of the new customer service conversationrecord to the respective centers of the K conversational cluster centersof the clusters. If the respective distances of the new customer serviceconversation record to the respective centers of the K conversationalcluster centers of the clusters are all greater than a threshold, thenthe new customer service conversation record is not considered to belongto any of the existing clusters of customer service conversation recordsand one or more appropriate management actions may be initiated (e.g.,creating a new cluster, raising one or more alerts, and so forth). Ifany of the respective distances of the new customer service conversationrecord to the respective centers of the K conversational cluster centersof the clusters is less than a threshold, then the currentconversation-level topic distribution vector may be updated. The currentconversation-level topic distribution vector may be updated using amultivariate anomaly detection algorithm (e.g., a multivariate changepoint detection algorithm or other suitable algorithm) that candetermine if the current conversational-level topic distribution vectorhas a significant difference from the previous conversational-leveltopic distribution vector. If the current conversational-level topicdistribution vector has a significant difference from the previousconversational-level topic distribution vector (e.g., measured based ona threshold or other suitable basis), one or more appropriate managementactions may be initiated (e.g., raising one or more alerts indicativethat there is a change in the topic distribution and so forth).

It will be appreciated that the customer service automation supportsystem 135 may be configured to support various other functions forsupporting automation of customer service provided to customers 101 bythe customer service center 130.

It will be appreciated that the customer service center 130 may beconfigured to support various other functions for supporting automationof customer service for customers 101.

It will be appreciated that the system 100 has been simplified and,thus, may be implemented in a different form than that which isillustrated in FIG. 1 . For example, system 100 may include additionalelements (e.g., networks, devices, and so forth), omit various elements,substitute elements for elements or devices that perform the same orsimilar functions, combine elements which are illustrated as separateelements, implement elements as functions that are spread across severalelements that operate collectively as the respective elements, and soforth, without altering the scope of the present disclosure. It will beappreciated that the system 100 may be modified in various other wayswhile still supporting automation of customer service for customers 101of customer service center 130 in accordance with the presentdisclosure. Thus, these and other modifications are contemplated withinthe scope of the present disclosure.

It will be appreciated that various functions for supporting automationof customer service based on analysis of customer service conversationsas discussed within the context of system 100 of FIG. 1 may be furtherunderstood by way of reference to FIGS. 2 - 5 , which may be implementedindependently or in various combinations.

FIG. 2 illustrates a flowchart of an example method for supportingautomation of customer service based on analysis of customer serviceconversations between customers and customer service agents. In oneexample, the method 200 is performed by one or more components of thesystem 100 of FIG. 1 (e.g., customer service automation support system135), one or more elements (e.g., a processor, or processors, performingoperations stored in and loaded from a memory) of one or more componentsof the system 100 of FIG. 1 , one or more components of the system 100of FIG. 1 in conjunction with one or more other devices, and so forth.In one example, the steps, functions, and/or operations of method 200may be performed by a computing system 600 as described in connectionwith FIG. 6 . For instance, the computing system 600 may represent anyone or more components of the system 100 of FIG. 1 , or portion(s)thereof, which is/are configured to perform the steps, functions, and/oroperations of the method 200. Similarly, in one example, the steps,functions, and/or operations of the method 200 may be performed by aprocessing system including one or more computing devices collectivelyconfigured to perform various steps, functions, and/or operations of themethod 200 (e.g., multiple instances of the computing system 600 maycollectively function as a processing system for performing varioussteps, functions, and/or operations of the method 200). It will beappreciated that the method 200 may be provided in various other ways.

As illustrated in FIG. 2 , the method 200 begins in step 205 andproceeds to step 210.

At step 210, the processing system may obtain a set of customer serviceconversation records for a set of customer service conversations betweena set of customers and a set of customer service agents. The customerservice conversations may be related to customer service provided to thecustomers by the customer service agents. The customer serviceconversation records may include audio records of voice conversations,text records of text-based conversations, and so forth.

At step 220, the processing system may perform an analysis of the set ofcustomer service conversation records. The analysis of the set ofcustomer service conversation records may include identification oftopics from the set of customer service conversation records (e.g.,conversation-level topics identified from the respective customerservice conversation records, conversation-level topics identified basedon clustering of the set of customer service conversation records intoclusters, primary conversation-level topics identified based onclustering of the set of customer service conversation records intoclusters, utterance-level topics identified from the respective customerservice conversation records, utterance-level topics identified based onclustering of the set of customer service conversation records intoclusters, and so forth). The analysis of the set of customer serviceconversation records may include determining a quantitative ranking of aset of topics from the set of customer service conversations, where theset of topics from the set of customer service conversations may includea set of conversation-level topics, a set of utterance-level topics, andso forth. In one example, the quantitative ranking of the set of topicsfrom the set of customer service conversations, for a set ofconversation-level topics, may be determined as presented with respectto FIG. 3 (although it will be appreciated that various operations ofFIG. 3 also may be used for other types of analysis of the set ofcustomer service conversation records). In one example, the quantitativeranking of the set of topics from the set of customer serviceconversations, for a set of utterance-level topics, may be determined aspresented with respect to FIG. 4 (although it will be appreciated thatvarious operations of FIG. 4 also may be used for other types ofanalysis of the set of customer service conversation records). It willbe appreciated that the quantitative ranking of the set of topics fromthe set of customer service conversations may be determined in otherways. It will be appreciated that the processing system may analyze theset of customer service conversation records in various other ways, mayanalyze the set of customer service conversation records to producevarious other outputs, and so forth.

At step 230, the processing system may initiate, based on the analysisof the set of customer service conversation records, a managementaction. The management action that is initiated may depend on the typeof analysis performed on the set of customer service conversationrecords, the output or outputs of the analysis of the set of customerservice conversation records, and so forth. The management action thatis initiated may include providing results of the analysis of the set ofcustomer service conversation records (e.g., sending the results to oneor more devices, causing the results to be presented, and so forth),generating information based on results of the analysis of the set ofcustomer service conversation records (e.g., generating one or morerecommendations for actions to be performed, generating software, and soforth), and so forth.

As illustrated in FIG. 2 , following step 230, the method 200 proceedsto step 295 where the method 200 ends.

It will be appreciated that the method 200 may be expanded to includeadditional steps, or may be modified to replace steps with differentsteps, to combine steps, to omit steps, to perform steps in a differentorder, and so forth. For instance, in one example the processing systemmay repeat one or more steps of the method 200 (e.g., obtaining andprocessing additional customer service conversation records for purposesof supporting various functions which may be supported based on analysisof the customer service conversations of the customer serviceconversation records). Thus, it will be appreciated that these and othermodifications are all contemplated within the scope of the presentdisclosure.

FIG. 3 illustrates a flowchart of an example method for determining aquantitative ranking of conversation-level topics from customer serviceconversations between customers and customer service agents. The method300 may be configured to support conversation-level repetitive topicextraction and ranking. In one example, the method 300 is performed byone or more components of the system 100 of FIG. 1 (e.g., customerservice automation support system 135), one or more elements (e.g., aprocessor, or processors, performing operations stored in and loadedfrom a memory) of one or more components of the system 100 of FIG. 1 ,one or more components of the system 100 of FIG. 1 in conjunction withone or more other devices, and so forth. In one example, the steps,functions, and/or operations of method 300 may be performed by acomputing system 600 as described in connection with FIG. 6 below. Forinstance, the computing system 600 may represent any one or morecomponents of the system 100 of FIG. 1 , or portion(s) thereof, whichis/are configured to perform the steps, functions, and/or operations ofthe method 300. Similarly, in one example, the steps, functions, and/oroperations of method 300 may be performed by a processing systemincluding one or more computing devices collectively configured toperform various steps, functions, and/or operations of the method 300(e.g., multiple instances of the computing system 600 may collectivelyfunction as a processing system for performing various steps, functions,and/or operations of the method 300). It will be appreciated that themethod 300 may be provided in various other ways.

As indicated above, method 300 of FIG. 3 may be performed as, or as partof, step 220 of FIG. 2 for conversation-level topics. Accordingly, priorto execution of method 300, it is assumed that the processing system hasobtained a set of customer service conversation records, for a set ofcustomer service conversations between a set of customers and a set ofcustomer service agents (e.g., as described with respect to step 210 ofmethod 200 of FIG. 2 or in any other suitable manner), which may beprocessed as described with respect to method 300 of FIG. 3 fordetermining a quantitative ranking of conversation-level topics from thecustomer service conversations between customers and customer serviceagents. Accordingly, it will be appreciated that step 210 may beincorporated within method 300 of FIG. 3 and, similarly, that varioussteps of method 300 of FIG. 3 may be included within method 200 of FIG.2 .

As illustrated in FIG. 3 , the method 300 begins in step 305 andproceeds to step 310.

At step 310, which is an optional step, the processing system mayconvert, for each of the customer service conversation records that isan audio customer service conversation record, the audio customerservice conversation record to a textual customer service conversationrecord. The conversion of the audio customer service conversationrecord(s) to textual customer service conversation record(s) may beperformed using one or more audio-to-text conversion programs orcapabilities. It will be appreciated that this conversion may beperformed for any audio customer service conversation records includedin the set of customer service conversation records (e.g., where some orall of the customer service conversations between the customers and thecustomer service agents are voice calls), but would not be performedwhere each of the customer service conversation records included in theset of customer service conversation records is a textual customerservice conversation record (e.g., where the customer serviceconversations between the customers and the customer service agents aretext-based conversations via chat interfaces).

At step 320, the processing system may generate a set of conversationaldocuments including, for each customer service conversation record inthe set of customer service conversation records, a respectiveconversational document. The conversational document for a customerservice conversation record may include a set of topic-relatedutterances from the customer service conversation. In one example, inwhich the customer service conversation records include topic-relatedutterances (e.g., associated with topics related to the customer serviceprovided to the customers by the customer service agents) andnon-topic-related utterances (e.g., greetings, chitchat, and so forth),generation of the conversational documents may include performingpre-cleaning of the customer service conversation records to removenon-topic-related utterances such that only topic-related utterancesremain. In one example, the processing system may generate theconversational document from a customer service conversation record byperforming, based on the customer service conversation record, anutterance pre-cleaning operation for removing from the customer serviceconversation record one or more non-topic-related utterances of therespective customer service conversation and maintaining thetopic-related utterances from the respective customer serviceconversation and concatenating the topic-related utterances from therespective customer service conversation to form thereby the respectiveconversational document for the respective customer service conversationrecord. In one example, utterance pre-cleaning for removingnon-topic-related utterances while maintaining topic-related utterancesmay be provided by an utterance pre-cleaning module. It will beappreciated that, by filtering such “noisy” non-topic-related utterancesthat could have a negative impact on further processing of theinformation of the customer service conversation records foridentification and ranking of topics for supporting customer serviceautomation, further processing of the topic-related utterances of thecustomer service conversations for identification and quantitativeranking of topics for supporting customer service automation is therebyimproved.

At step 330, the processing system may generate a set of conversationaldocument representations including, for each conversational document inthe set of conversational documents, a respective conversationaldocument representation of the respective conversational document. Inone example, the conversational document representations of theconversational documents may be neural representations of theconversational documents. In one example, the neural representations ofthe conversational documents may be lower-dimensional neuralrepresentations of the conversational documents. In one example, theneural representations of the conversational documents may be generatedby training a deep neural network based on the conversational documents.In one example, the conversational document representations may begenerated based on feature extraction, such as based on use of aconversation-level neural feature extractor configured to performfeature extraction. It will be appreciated that the conversationaldocument representations of the conversational documents may begenerated in various other ways.

At step 340, the processing system may divide the conversationaldocument representations in the set of conversational documentrepresentations into a set of clusters, wherein each of the clustersrepresents a respective group of the customer service conversationshaving a respective topic in common. In one example, the clustering ofthe conversational document representations into the clusters may beperformed based on a clustering algorithm. In one example, theclustering of the conversational document representations into theclusters may be performed in a manner for grouping conversationaldocument representations of conversational documents associated withcustomer service conversations having a respective topic in common, suchthat this also may be considered to be a clustering of theconversational documents associated with customer service conversationsor a clustering of the customer service conversations. It will beappreciated that the topic in common for a given cluster ofconversational document representations may or may not be theconversation-level topic that is ultimately identified for the cluster(e.g., based on use of topic modeling or extraction algorithms asdiscussed further below), since the various customer serviceconversations represented by the conversational document representationseach may include utterances covering a range of topics. It will beappreciated that, since the customer service conversations each mayinclude utterances covering a range of topics, the clustering of theconversational document representations of the conversational documentsassociated with the customer service conversations may be performed in amanner tending to support identification of conversation-level topics(e.g., selection of common topics as a basis for clustering may be basedon knowledge about the topic modeling or extraction algorithms may beapplied for identification of the conversation-level topics from varioussets of topics covered in the various customer service conversationsrepresented by the conversational document representations in theclusters, respectively) or may be performed independent of attempting tosupport identification of conversation-level topics (e.g., after theclusters are determined then topic modeling or extraction algorithms maybe applied for identification of the conversation-level topics fromvarious sets of topics covered in the various customer serviceconversations represented by the conversational document representationsin the clusters, respectively).

At step 350, the processing system may determine a set ofconversation-level topics including, for each cluster in the set ofclusters, a respective conversation-level topic for the respectivecluster that is determined based on the respective conversationaldocument representations in the respective cluster.

In one example, the processing system may determine theconversation-level topics for the respective clusters based on one ormore algorithms, models, and so forth. In one example, the processingsystem may determine the conversation-level topics for the respectiveclusters based on a topic extraction algorithm. In one example, theprocessing system may determine the conversation-level topics for therespective clusters based on a generative statistical model. In oneexample, the generative statistical model includes an LDA model. It willbe appreciated that the processing system may determine theconversation-level topics for the respective clusters based on variousother algorithms, models, and so forth.

In one example, for one or more of the clusters, the processing systemmay determine the conversation-level topic for the cluster based onprocessing of the respective conversational document representations inthe respective cluster for identification of topic-related utterances inthe customer service conversations (e.g., relatively important words orphrases) in the group of the customer service conversations having therespective topic in common for the respective cluster. Here, it is notedthat, since each customer service conversation may include multipletopics, some of which are irrelevant to the main topic of the clusterand others which are relevant to the main topic of the cluster,filtering of topics which are irrelevant to the main topic of thecluster may result in identification of the topics that are relevant tothe main topic of the cluster, which may then be further evaluated(e.g., based on a topic extraction algorithm, such as based on an LDAmodel) to determine the relevant topic that is considered to be theconversation-level topic for the customer service conversationsrepresented by the conversational document representations in therespective cluster.

It will be appreciated that the set of conversation-level topics mayrepresent the most repetitive conversation-level topics from the set ofcustomer service conversation records of the customer serviceconversations.

At step 360, the processing system may determine, based on the clusters,a quantitative ranking of the conversation-level topics in the set ofconversation-level topics. In one example, the quantitative ranking ofthe conversation-level topics in the set of conversation-level topicsmay be determined based on respective cluster sizes of the respectiveclusters with which the respective conversation-level topics areassociated (e.g., larger cluster sizes may result in higher quantitativerankings of the respective conversation-level topics associated with theclusters and smaller cluster sizes may result in lower quantitativerankings of the respective conversation-level topics associated with theclusters). In one example, the quantitative ranking of theconversation-level topics in the set of conversation-level topics may bedetermined based on a distribution of the respective cluster sizes ofthe respective clusters with which the respective conversation-leveltopics are associated. In one example, the cluster size may be based onor indicative of the number of customer service conversations associatedwith the cluster (e.g., customer service conversations represented byconversational document representations included in the cluster). Itwill be appreciated that the quantitative ranking of theconversation-level topics in the set of conversation-level topics may bedetermined in other ways.

As illustrated in FIG. 3 , following step 360, the method 300 proceedsto step 395 where the method 300 ends.

It will be appreciated that the method 300 may be expanded to includeadditional steps, or may be modified to replace steps with differentsteps, to combine steps, to omit steps, to perform steps in a differentorder, and so forth. For instance, in one example the processing systemmay repeat one or more steps of the method 300 (e.g., obtaining andprocessing additional customer service conversation records for purposesof determining a new quantitative ranking of conversation-level topicsfrom additional customer service conversations between customers andcustomer service agents). Thus, it will be appreciated that these andother modifications are all contemplated within the scope of the presentdisclosure.

It will be appreciated that, although primarily described with respectto performing method 300 of FIG. 3 within a particular context (e.g.,within the context of method 200 of FIG. 2 (e.g., where a quantitativeranking of conversation-level topics is determined and used as a basisfor initiating one or more management actions)), in at least someexamples the method 300 of FIG. 3 may be modified such that thequantitative ranking of conversation-level topics is not determined(e.g., step 360 of method 300 is not performed), such that the result ofmethod 300 is identification of the set of conversation-level topics(e.g., the highly repetitive conversation-level topics, the mostimportant conversation-level topics, and the like). It will beappreciated that the list of conversation-level topics may then behandled or used in various ways which may or may not includedetermination of quantitative rankings of conversation-level topics,initiating of management actions based on the conversation-level topics,and so forth. It will be appreciated that method 300 of FIG. 3 may bemodified in various other ways (e.g., including additional steps,removing or modifying existing steps, and the like), used within variousother contexts (e.g., to support various other methods, within variousother environments, for various other purposes, and the like), and soforth.

FIG. 4 illustrates a flowchart of an example method for determining aquantitative ranking of utterance-level topics from conversationsbetween customers and customer service agents. The method 400 may beconfigured to support utterance-level repetitive topic extraction andranking. The method 400 may be configured to support fine-grainedutterance-level repetitive topic extraction and ranking from aconversational cluster identified from a set of customer serviceconversation records (e.g., customer service conversation recordsassociated with a conversation-level topic). In one example, the method400 is performed by one or more components of the system 100 of FIG. 1(e.g., customer service automation support system 135), one or moreelements (e.g., a processor, or processors, performing operations storedin and loaded from a memory) of one or more components of the system 100of FIG. 1 , one or more components of the system 100 of FIG. 1 inconjunction with one or more other devices, and so forth. In oneexample, the steps, functions, and/or operations of method 400 may beperformed by a computing system 600 as described in connection with FIG.6 . For instance, the computing system 600 may represent any one or morecomponents of the system 100 of FIG. 1 , or portion(s) thereof, whichis/are configured to perform the steps, functions, and/or operations ofthe method 400. Similarly, in one example, the steps, functions, and/oroperations of the method 400 may be performed by a processing systemincluding one or more computing devices collectively configured toperform various steps, functions, and/or operations of the method 400(e.g., multiple instances of the computing system 600 may collectivelyfunction as a processing system for performing various steps, functions,and/or operations of the method 400). It will be appreciated that themethod 400 may be provided in various other ways.

As indicated above, method 400 of FIG. 4 may be performed as, or as partof, step 220 of FIG. 2 . Accordingly, prior to execution of method 400,it is assumed that the processing system has obtained a set of customerservice conversation records, for a set of customer serviceconversations between a set of customers and a set of customer serviceagents (e.g., as described with respect to step 210 of method 200 ofFIG. 2 or in any other suitable manner), which may be processed fordetermining a quantitative ranking of utterance-level topics from arelated set of customer service conversations between customers andcustomer service agents (e.g., from a cluster of customer serviceconversations which, as indicated above and discussed further below, maybe determined during execution of method 300 of FIG. 3 or based on useof a portion of method 300 of FIG. 3 ). Accordingly, it will beappreciated that step 210 may be incorporated within method 400 of FIG.4 and that one or more steps of method 300 may be incorporated withinmethod 400 of FIG. 4 , and, similarly, that various steps of method 400of FIG. 4 may be included within method 300 of FIG. 3 and/or withinmethod 200 of FIG. 2 .

As illustrated in FIG. 4 , the method 400 begins in step 405 andproceeds to step 410.

At step 410, the processing system may obtain, from a cluster ofcustomer service conversations associated with a conversation-leveltopic, a set of utterances associated with the conversation-level topic.The conversation-level topic may be a conversation-level topic in commonfor the cluster of customer service conversations. Theconversation-level topic may be a primary conversation-level topic forthe cluster of customer service conversations. In one example, the setof utterances may be obtained by selecting one of the clusters generatedbased on execution of method 300 of FIG. 3 . In one example, the set ofutterances may be obtained based on execution of a portion of the stepsof method 300 of FIG. 3 (e.g., steps 310-340) to obtain a cluster ofcustomer service conversations associated with the conversation-leveltopic and, thus, to obtain the utterances from the cluster of customerservice conversations associated with the conversation-level topic.

At step 420, the processing system may obtain a business-specificvocabulary related to the conversation-level topic. Thebusiness-specific vocabulary may include words and phrases that are ormay be relevant under various scenarios, such as for specific types ofentities, specific types of products or services for which customerservice is provided, specific types of customer service, specificconversation-level topics, and so forth. For example, for a call centerfor technical support for a telecommunications company, thebusiness-specific vocabulary may include words such as “router,”“gateway,” and so forth. For example, for a service center for orderingsupport for an e-commerce company, the business-specific vocabulary mayinclude words such as “account,” “shopping cart,” “credit card,” and soforth. The business-specific vocabulary may be predefined by experts,defined over time based on analysis of customer service conversationrecords, and so forth.

At step 430, the processing system may extract, from the set ofutterances associated with the conversation-level topic based on thebusiness-specific vocabulary, a set of business-specific phrases. In oneexample, the extraction of the set business-specific phrases may includeextracting phrases with n words that include at least one word from thebusiness-specific vocabulary. In one example, n may be a tunableparameter.

At step 440, the processing system may extract, from the set ofutterances associated with the conversation-level topic based on the setof business-specific phrases, a set of utterance-level topics. In oneexample, the processing system may extract the set of utterance-leveltopics from the set of utterances associated with the conversation-leveltopic based on a topic extraction algorithm. In one example, theprocessing system may extract the set of utterance-level topics from theset of utterances associated with the conversation-level topic based ona generative statistical model. In one example, the generativestatistical model includes an LDA model. It is noted that use of thebusiness-specific phrases restricts the generative statistical model(e.g., the LDA model) in a manner that supports filtering of irrelevantterms and identification of instructive topics. In one example, theprocessing system may extract the set of utterance-level topics from theset of utterances associated with the conversation-level topic by, foreach utterance in the set of utterances associated with theconversation-level topic, determining a probability distribution oftopics of the respective utterance (e.g., for topics which are composedof business-specific phrases from the set of business-specific phrases)and assigning the topic with the largest probability to the respectiveutterance as the respective utterance-level topic for the respectiveutterance. It will be appreciated that the set of utterance-level topicsfor the respective utterances may represent the most repetitiveutterance-level topics from the conversation-level topic (and, thus,from the set of customer service conversation records associated withthe conversation-level topic).

At step 450, the processing system may determine a quantitative rankingof the utterance-level topics in the set of utterance-level topics. Inone example, the quantitative ranking of the utterance-level topics inthe set of utterance-level topics may be based on the respectiveprobabilities of the respective utterance-level topics in the set ofutterance-level topics (e.g., based on the determination of theprobabilities of the topics in determining the probability distributionsof topics of the respective utterances as described with respect to step440). It will be appreciated that the quantitative ranking of theutterance-level topics in the set of utterance-level topics may bedetermined in other ways.

As illustrated in FIG. 4 , following step 450, the method 400 proceedsto step 495 where the method 400 ends.

It will be appreciated that the method 400 may be expanded to includeadditional steps, or may be modified to replace steps with differentsteps, to combine steps, to omit steps, to perform steps in a differentorder, and so forth. For instance, in one example the processing systemmay repeat one or more steps of the method 400 (e.g., repeating method400 for one or more other sets of utterances associated with one or moreother conversation-level topics (e.g., for one or more other clustersidentified during execution of method 300 of FIG. 3 )). Thus, it will beappreciated that these and other modifications are all contemplatedwithin the scope of the present disclosure.

It will be appreciated that, although primarily described with respectto performing method 400 of FIG. 4 within a particular context (e.g.,within the context of method 200 of FIG. 2 (e.g., where a quantitativeranking of utterance-level topics is determined and used as a basis forinitiating one or more management actions)), in at least some examplesmethod 400 of FIG. 4 may be modified such that the quantitative rankingof utterance-level topics is not determined (e.g., step 450 of method400 is not performed), such that the result of method 400 isidentification of the set of utterance-level topics (e.g., the highlyrepetitive utterance-level topics, the most important utterance-leveltopics, and the like). It will be appreciated that the list ofutterance-level topics may then be handled or used in various ways whichmay or may not include determination of quantitative rankings of theutterance-level topics, initiating of management actions based on theutterance-level topics, and so forth. It will be appreciated that method400 of FIG. 4 may be modified in various other ways (e.g., includingadditional steps, removing or modifying existing steps, and the like),used within various other contexts (e.g., to support various othermethods, within various other environments, for various other purposes,and the like), and so forth.

FIG. 5 illustrates a flowchart of an example method for initiating acustomer service automation action based on analysis of a set ofcustomer service conversation records. In one example, the method 500 isperformed by one or more components of the system 100 of FIG. 1 (e.g.,customer service automation support system 135), one or more elements(e.g., a processor, or processors, performing operations stored in andloaded from a memory) of one or more components of the system 100 ofFIG. 1 , one or more components of the system 100 of FIG. 1 inconjunction with one or more other devices, and so forth. In oneexample, the steps, functions, and/or operations of method 500 may beperformed by a computing system 600 as described in connection with FIG.6 . For instance, the computing system 600 may represent any one or morecomponents of the system 100 of FIG. 1 , or portion(s) thereof, whichis/are configured to perform the steps, functions, and/or operations ofthe method 500. Similarly, in one example, the steps, functions, and/oroperations of the method 500 may be performed by a processing systemincluding one or more computing devices collectively configured toperform various steps, functions, and/or operations of the method 500(e.g., multiple instances of the computing system 600 may collectivelyfunction as a processing system for performing various steps, functions,and/or operations of the method 500). As indicated above, method 500 ofFIG. 5 may be performed as, or as part of, method 200 of FIG. 2 . Itwill be appreciated that the method 500 may be provided in various otherways.

As illustrated in FIG. 5 , the method 500 begins in step 505 andproceeds to step 510.

At step 510, the processing system obtain a set of customer serviceconversation records, wherein each of the customer service conversationrecords includes a respective set of utterances. The customer serviceconversation records may be related to customer service provided to thecustomers by the customer service agents. The customer serviceconversation records may include audio records of voice conversations,text records of text-based conversations, and so forth. The customerservice conversation records may include various utterances related tovarious conversation-level topics, various utterance-level topics, andso forth.

At step 520, the processing system may divide the set of customerservice conversation records into a set of clusters, wherein each of theclusters represents a respective group of the customer serviceconversation records having a respective conversation-level topic incommon. In one example, dividing the set of customer serviceconversation records into the set of clusters may include generating aset of conversational documents including, for each customer serviceconversation record in the set of customer service conversation records,a respective conversational document, generating a set of conversationaldocument representations including, for each conversational document inthe set of conversational documents, a respective conversationaldocument representation of the respective conversational document, anddividing, based on a clustering algorithm, the set of conversationaldocument representations into the set of clusters. In one example, theset of conversational document representations includes a respective setof neural representations of the respective conversational documents. Inone example, the neural representations of the respective conversationaldocuments are generated based on at least one of a training of a deepneural network and a conversation-level neural feature extractor. In oneexample, for at least one of the customer service conversation records,the respective set of utterances includes a respective set oftopic-related utterances and a respective set of non-topic-relatedutterances, and generating the respective conversational documentincludes performing, for the respective customer service conversationrecord, an utterance pre-cleaning operation for removing the respectiveset of non-topic-related utterances from the respective customer serviceconversation record and maintaining the respective set of topic-relatedutterances within the respective customer service conversation recordand concatenating, for the respective customer service conversationrecords, the respective topic-related utterances in the set oftopic-related utterances to form the respective conversational document.

At step 530, the processing system may determine, from the set ofclusters, a set of primary conversation-level topics. In one example,the set of primary conversation-level topics may include one primaryconversation-level topic from each of the clusters, may include primaryconversation-level topics from only a subset of the clusters, mayinclude multiple primary conversation-level topics from one or more ofthe clusters, and so forth. In one example, determining the set ofprimary conversation-level topics includes determining, from at leastone of the clusters based application of a topic extraction algorithm tothe utterances of the at least one of the clusters, a respective primaryconversation-level topic for the at least one of the clusters. In oneexample, the topic extraction algorithm is based on an LDA model. In oneexample, for at least one of the clusters in the set of clusters, arespective primary conversation-level topic determined for the at leastone of the clusters may be different than the respectiveconversation-level topic in common for the at least one of the clusters(i.e., the topic which causes the customer service conversation recordsmay not necessarily be the conversation-level topic that is extractedfrom the customer service conversation records as the primaryconversation-level topic for the respective cluster). In one example,the at least one of the primary conversation-level topics is selectedbased on a quantitative ranking of the primary conversation-level topicsin the set of primary conversation-level topics.

At step 540, the processing system may obtain, for at least one of theprimary conversation-level topics associated with at least one of theclusters, a set of keywords. The set of keywords may be obtained basedon the at least one of the primary conversation-level topics. The set ofkeywords may be obtained based on the at least one of the primaryconversation-level topics since the set of keywords that is used toextract utterance-level topics may vary across different primaryconversation-level topics. The set of keywords may be obtained from asource different than the utterances of the at least one of the clustersof the at least one of the primary conversation-level topics. The set ofkeywords may be obtained from a source different than the set ofclusters (e.g., independent of the various utterances of the customerservice conversation records grouped into the clusters). The source maybe any suitable source of such information (e.g., a file that isindependent of the customer service conversation records or utterancesfrom the customer service conversation records, a memory, a system, andso forth). The set of keywords may be based on and/or include abusiness-specific vocabulary related to the primary conversation-leveltopic (e.g., words and/or phrases that are or may be relevant undervarious scenarios, such as for specific types of entities, specifictypes of products or services for which customer service is provided,specific types of customer service, specific conversation-level topics,and so forth), which may be predefined by experts, defined over timebased on analysis of customer service conversation records, and soforth.

At step 550, the processing system may determine, from the utterances ofthe at least one of the clusters of the at least one of the primaryconversation-level topics, a set of utterance-level topics associatedwith the at least one of the primary conversation-level topics. The setof utterance-level topics may be determined based on the set ofkeywords. In one example, determining the set of utterance-level topicsassociated with the at least one of the primary conversation-leveltopics includes extracting, from the utterances of the at least one ofthe clusters of the at least one of the primary conversation-leveltopics, a set of phrases, wherein the set of phrases is extracted basedon the set of keywords, and extracting, from the utterances of the atleast one of the clusters of the at least one of the primaryconversation-level topics, the set of utterance-level topics, whereinthe set of utterance-level topics is extracted based on the set ofphrases and using a topic extraction algorithm. In one example, the setof keywords includes a business-specific vocabulary and the set ofphrases includes a set of business-specific phrases. In one example, thetopic extraction algorithm is based on an LDA model.

At step 560, the processing system may initiate, based on the at leastone of the primary conversation-level topics and the set ofutterance-level topics, a customer service automation action. In oneexample, the customer service automation action includes one or more ofproviding a notification, providing a recommendation, generatingsoftware, and so forth. In one example, the customer service automationaction includes providing a recommendation for a creation of a chatbotthat is based on the at least one of the primary conversation-leveltopics and a recommendation for a design of the chatbot that is based onthe set of utterance-level topics associated with the at least one ofthe primary conversation-level topics.

As illustrated in FIG. 5 , following step 560, the method 500 proceedsto step 595 where the method 500 ends.

It will be appreciated that the method 500 may be expanded to includeadditional steps, or may be modified to replace steps with differentsteps, to combine steps, to omit steps, to perform steps in a differentorder, and so forth. For instance, in one example the processing systemmay repeat one or more steps of the method 500 (e.g., repeating all orpart of method 500 for one or more other primary conversation-leveltopics). Thus, it will be appreciated that these and other modificationsare all contemplated within the scope of the present disclosure.

It will be appreciated that, although not expressly specified above, oneor more steps of one or more of the method 200, the method 300, themethod 400, and/or the method 500 may include a storing, displaying,and/or outputting step as required for a particular application. Inother words, any data, records, fields, and/or intermediate resultsdiscussed in the method(s) can be stored, displayed, and/or outputted toanother device as required for a particular application. Furthermore,operations, steps, or blocks in one or more of the method 200, themethod 300, the method 400, and/or the method 500 that recite adetermining operation or involve a decision do not necessarily requirethat both branches of the determining operation be practiced. In otherwords, one of the branches of the determining operation can be deemed asan optional step. However, the use of the term “optional step” isintended to only reflect different variations of a particularillustrative example and is not intended to indicate that steps notlabelled as optional steps to be deemed to be essential steps.Furthermore, operations, steps, or blocks in one or more of the method200, the method 300, the method 400, and/or the method 500 can becombined, separated, and/or performed in a different order from thatdescribed above, without departing from the examples of the presentdisclosure.

It will be appreciated that various examples for supporting automationof customer service as presented herein may provide various advantagesor potential advantages. For example, various examples for supportingautomation of customer service may support an advanced data-drivenapproach for customer service center automation strategic planning anddesign. For example, various examples for supporting automation ofcustomer service may extract repetitive conversational topics and/orutterance-level topics from customer service conversation records anduse the extracted topics to support customer service center automation.For example, various examples for supporting automation of customerservice may provide improved automation of customer service based on useof topic modeling and neural features. For example, various examples forsupporting automation of customer service may provide a more efficientand scalable data-driven approach for identifying and prioritizingautomation options for customer service center automation without havingto rely on experts familiar with business and operational aspects of theentities for which customer service is being provided. For example,various examples for supporting automation of customer service mayenable mining of customer service data, such as customer phone callrecords and online chat records, for various purposes (e.g., monitoringproduct/service quality, problem/issue tracking, and so forth). Forexample, various examples for supporting automation of customer servicemay obviate the need for use of generic chatbots for handling multipletasks by using analysis of customer service conversation records toidentify specific topics for which more specific and intelligentchatbots (e.g., more accurate and reliable) may be designed (e.g., forhandling specialized tasks such as cancelling customer orders,responding to requests for autoinsurance quotes, and the like). Forexample, various examples for supporting automation of customer servicemay provide an automatic repetitive topic extraction system to supportautomation of specific customer service situations and functions (e.g.,determining which types of customer service requests should have thehighest automation priority, determining the design of theconversational dialog of a chatbot(s) to solve specific customer servicerequirements, and so forth). For example, various examples forsupporting automation of customer service may use advanced naturallanguage processing and machine learning techniques to quantitativelyrank the importance of conversation topics of customer serviceinteractions and to prioritize customer service center automation tasks,thereby enabling data-driven automation decisions and significantlyreducing the cost of human labor wasted in repetitive customer servicework by supporting automation of various customer service center tasksrelated to greater numbers of customer service agent interactions (e.g.,if some call center agents take care of phone calls about order statustracking while other call center agents handle phone calls abouttroubleshooting malfunctioning devices, and the former task incurs ahigher number of phone calls than the latter task, then it may be morebeneficial (e.g., from a business cost reduction perspective), toautomate the former task before automating the latter task). Forexample, various examples for supporting automation of customer servicemay support quantitative ranking of automation priority without humanassistance to inspect the audio records of phone calls, may supportquantitative ranking of automation priority within the context ofcustomer service involving a large number of distinctive tasks, and soforth. It will be appreciated that various examples for supportingautomation of customer service as presented herein may provide variousother advantages or potential advantages.

It will be appreciated that, as used herein, the terms “configure,” and“reconfigure” may refer to programming or loading a processing systemwith computer-readable/computer-executable instructions, code, and/orprograms, e.g., in a distributed or non-distributed memory, which whenexecuted by a processor, or processors, of the processing system withina same device or within distributed devices, may cause the processingsystem to perform various functions. Such terms may also encompassproviding variables, data values, tables, objects, or other datastructures and the like which may cause a processing system executingcomputer-readable instructions, code, and/or programs to functiondifferently depending upon the values of the variables or other datastructures that are provided. As referred to herein a “processingsystem” may include a computing device, or computing system, includingone or more processors, or cores (e.g., as illustrated in FIG. 6 anddiscussed below) or multiple computing devices collectively configuredto perform various steps, functions, and/or operations in accordancewith the present disclosure.

FIG. 6 depicts a high-level block diagram of a computing system 600(e.g., a computing device or a processing system) specificallyprogrammed to perform functions described herein. For example, any oneor more components or devices illustrated in FIG. 1 or described inconnection with method 200 of FIG. 2 , method 300 of FIG. 3 , method 400of FIG. 4 , or method 500 of FIG. 5 may be implemented as the computingsystem 600. As depicted in FIG. 6 , the computing system 600 includes ahardware processor element 602 (e.g., including one or more hardwareprocessors, which may include one or more microprocessor(s), one or morecentral processing units (CPUs), and/or the like, where the hardwareprocessor element 602 also may represent one example of a “processingsystem” as referred to herein), a memory 604 (e.g., random access memory(RAM), read only memory (ROM), a disk drive, an optical drive, amagnetic drive, and/or a Universal Serial Bus (USB) drive), a module 605for supporting automation of customer service and/or any other functionor combination of functions presented herein, and one or moreinput/output devices 606 (e.g., a camera, a video camera, storagedevices, including but not limited to, a tape drive, a floppy drive, ahard disk drive, or a compact disk drive, a receiver, a transmitter, aspeaker, a display, a speech synthesizer, an output port, and a userinput device (such as a keyboard, a keypad, a mouse, and the like)).

It will be appreciated that, although only one hardware processorelement 602 is shown, the computing system 600 may employ a plurality ofhardware processor elements. Furthermore, although only one computingdevice is shown in FIG. 6 , if the method(s) as discussed above isimplemented in a distributed or parallel manner for a particularillustrative example, e.g., the steps of the above method(s) or theentire method(s) are implemented across multiple or parallel computingdevices, then the computing system 600 of FIG. 6 may represent each ofthose multiple or parallel computing devices. Furthermore, one or morehardware processor elements (e.g., hardware processor element 602) canbe utilized in supporting a virtualized or shared computing environment.The virtualized computing environment may support one or more virtualmachines which may be configured to operate as computers, servers, orother computing devices. In such virtualized virtual machines, hardwarecomponents such as hardware processors and computer-readable storagedevices may be virtualized or logically represented. The hardwareprocessor element 602 also may be configured or programmed to causeother devices to perform one or more operations as discussed above. Inother words, the hardware processor element 602 may serve the functionof a central controller directing other devices to perform the one ormore operations as discussed above.

It will be appreciated that the present disclosure can be implemented insoftware and/or in a combination of software and hardware, e.g., usingapplication specific integrated circuits (ASIC), a programmable logicarray (PLA), including a field-programmable gate array (FPGA), or astate machine deployed on a hardware device, a computing device, or anyother hardware equivalents, e.g., computer-readable instructionspertaining to the method(s) discussed above can be used to configure oneor more hardware processor elements to perform the steps, functionsand/or operations of the above disclosed method(s). In one example,instructions and data for the module 605 for supporting automation ofcustomer service and/or any other function or combination of functionspresented herein (e.g., a software program include computer-executableinstructions) can be loaded into memory 604 and executed by hardwareprocessor element 602 to implement the steps, functions and/oroperations as discussed above in connection with example method 200 ofFIG. 2 , example method 300 of FIG. 3 , example method 400 of FIG. 4 ,or example method 500 of FIG. 5 . Furthermore, when a hardware processorelement executes instructions to perform operations, this could includethe hardware processor element performing the operations directly and/orfacilitating, directing, or cooperating with one or more additionalhardware devices or components (e.g., a co-processor and the like) toperform the operations.

The processor (e.g., hardware processor element 602) executing thecomputer-readable instructions relating to the above described method(s)can be perceived as a programmed processor or a specialized processor.As such, the module 605 for supporting automation of customer serviceand/or any other function or combination of functions presented herein(including associated data structures) of the present disclosure can bestored on a tangible or physical (broadly non-transitory)computer-readable storage device or medium, e.g., volatile memory,non-volatile memory, ROM memory, RAM memory, magnetic or optical drive,device or diskette, and the like. Furthermore, a “tangible”computer-readable storage device or medium may comprise a physicaldevice, a hardware device, or a device that is discernible by the touch.More specifically, the computer-readable storage device or medium maycomprise any physical devices that provide the ability to storeinformation such as instructions and/or data to be accessed by aprocessor or a computing device such as a computer or an applicationserver.

While various examples have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. Thus, the breadth and scope of a preferred example shouldnot be limited by any of the above-described examples, but should bedefined only in accordance with the following claims and theirequivalents.

What is claimed is:
 1. A method comprising: obtaining, by a processingsystem including at least one processor, a set of customer serviceconversation records, wherein each of the customer service conversationrecords includes a respective set of utterances; dividing, by theprocessing system, the set of customer service conversation records intoa set of clusters, wherein each of the clusters represents a respectivegroup of the customer service conversation records having a respectiveconversation-level topic in common; determining, by the processingsystem from the set of clusters, a set of primary conversation-leveltopics; obtaining, by the processing system for at least one of theprimary conversation-level topics associated with at least one of theclusters, a set of keywords, wherein the set of keywords is obtainedbased on the at least one of the primary conversation-level topics,wherein the set of keywords is obtained from a source different than theset of clusters; determining, by the processing system from theutterances of the at least one of the clusters of the at least one ofthe primary conversation-level topics, a set of utterance-level topicsassociated with the at least one of the primary conversation-leveltopics, wherein the set of utterance-level topics is determined based onthe set of keywords; and initiating, by the processing system based onthe at least one of the primary conversation-level topics and the set ofutterance-level topics, a customer service automation action.
 2. Themethod of claim 1, wherein dividing the set of customer serviceconversation records into the set of clusters comprises: generating, bythe processing system, a set of conversational documents including, foreach customer service conversation record in the set of customer serviceconversation records, a respective conversational document; generating,by the processing system, a set of conversational documentrepresentations including, for each conversational document in the setof conversational documents, a respective conversational documentrepresentation of the respective conversational document; and dividing,by the processing system based on a clustering algorithm, the set ofconversational document representations into the set of clusters.
 3. Themethod of claim 2, wherein the set of conversational documentrepresentations comprises a respective set of neural representations ofthe respective conversational documents.
 4. The method of claim 3,wherein the neural representations of the respective conversationaldocuments are generated based on a training of a deep neural network. 5.The method of claim 3, wherein the neural representations of therespective conversational documents are generated based on aconversation-level neural feature extractor.
 6. The method of claim 1,wherein the determining the set of primary conversation-level topicscomprises: determining, from at least one of the clusters based on anapplication of a topic extraction algorithm to the utterances of the atleast one of the clusters, a respective primary conversation-level topicfor the at least one of the clusters.
 7. The method of claim 6, whereinthe topic extraction algorithm is based on a latent dirichlet allocationmodel.
 8. The method of claim 1, wherein, for at least one of theclusters in the set of clusters, a respective primary conversation-leveltopic determined for the at least one of the clusters is different thanthe respective conversation-level topic in common for the at least oneof the clusters.
 9. The method of claim 1, wherein the at least one ofthe primary conversation-level topics is selected based on aquantitative ranking of the primary conversation-level topics in the setof primary conversation-level topics.
 10. The method of claim 1, whereinthe determining the set of utterance-level topics associated with the atleast one of the primary conversation-level topics comprises:extracting, by the processing system from the utterances of the at leastone of the clusters of the at least one of the primaryconversation-level topics, a set of phrases, wherein the set of phrasesis extracted based on the set of keywords; and extracting, by theprocessing system from the utterances of the at least one of theclusters of the at least one of the primary conversation-level topics,the set of utterance-level topics, wherein the set of utterance-leveltopics is extracted based on the set of phrases and using a topicextraction algorithm.
 11. The method of claim 10, wherein the set ofkeywords comprises a business-specific vocabulary and the set of phrasescomprises a set of business-specific phrases.
 12. The method of claim10, wherein the topic extraction algorithm is based on a latentdirichlet allocation model.
 13. The method of claim 1, furthercomprising: determining, by the processing system, a quantitativeranking of the primary conversation-level topics in the set of primaryconversation-level topics.
 14. The method of claim 13, furthercomprising: obtaining, by the processing system, a second set ofcustomer service conversation records; determining, by the processingsystem based on the second set of customer service conversation records,a second quantitative ranking of a second set of primaryconversation-level topics extracted from the second set of customerservice conversation records; and evaluating, by the processing systembased on the quantitative ranking and the second quantitative ranking,the at least one of the primary conversation-level topics.
 15. Themethod of claim 13, further comprising: generating, by the processingsystem based on the quantitative ranking, an indication of a potentialproblem to be addressed.
 16. The method of claim 1, further comprising:determining, by the processing system, a quantitative ranking of theutterance-level topics in the set of utterance-level topics.
 17. Themethod of claim 16, wherein the customer service automation action isselected based on the quantitative ranking of the utterance-level topicsin the set of utterance-level topics.
 18. The method of claim 1, whereinthe customer service automation action comprises providing arecommendation for a creation of a chatbot that is based on the at leastone of the primary conversation-level topics and a recommendation for adesign of the chatbot that is based on the set of utterance-level topicsassociated with the at least one of the primary conversation-leveltopics.
 19. A non-transitory computer-readable storage medium storinginstructions which, when executed by a processing system, cause theprocessing system to perform operations, the operations comprising:obtaining a set of customer service conversation records, wherein eachof the customer service conversation records includes a respective setof utterances; dividing the set of customer service conversation recordsinto a set of clusters, wherein each of the clusters represents arespective group of the customer service conversation records having arespective conversation-level topic in common; determining, from the setof clusters, a set of primary conversation-level topics; obtaining, forat least one of the primary conversation-level topics associated with atleast one of the clusters, a set of keywords, wherein the set ofkeywords is obtained based on the at least one of the primaryconversation-level topics, wherein the set of keywords is obtained froma source different than the set of clusters; determining, from theutterances of the at least one of the clusters of the at least one ofthe primary conversation-level topics, a set of utterance-level topicsassociated with the at least one of the primary conversation-leveltopics, wherein the set of utterance-level topics is determined based onthe set of keywords; and initiating, based on the at least one of theprimary conversation-level topics and the set of utterance-level topics,a customer service automation action.
 20. An apparatus comprising: aprocessing system including at least one processor; and acomputer-readable medium storing instructions which, when executed bythe processing system, cause the processing system to performoperations, the operations comprising: obtaining a set of customerservice conversation records, wherein each of the customer serviceconversation records includes a respective set of utterances; dividingthe set of customer service conversation records into a set of clusters,wherein each of the clusters represents a respective group of thecustomer service conversation records having a respectiveconversation-level topic in common; determining, from the set ofclusters, a set of primary conversation-level topics; obtaining, for atleast one of the primary conversation-level topics associated with atleast one of the clusters, a set of keywords, wherein the set ofkeywords is obtained based on the at least one of the primaryconversation-level topics, wherein the set of keywords is obtained froma source different than the set of clusters; determining, from theutterances of the at least one of the clusters of the at least one ofthe primary conversation-level topics, a set of utterance-level topicsassociated with the at least one of the primary conversation-leveltopics, wherein the set of utterance-level topics is determined based onthe set of keywords; and initiating, based on the at least one of theprimary conversation-level topics and the set of utterance-level topics,a customer service automation action.